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Abstract 

We present a formulation of the problem of probabilistic model checking as one of query 
evaluation over probabilistic logic programs. To the best of our knowledge, our formulation is 
the first of its kind, and it covers a rich class of probabilistic models and probabilistic temporal 
logics. The inference algorithms of existing probabilistic logic-programming systems are well 
defined only for queries with a finite number of explanations. This restriction prohibits the 
encoding of probabilistic model checkers, where explanations correspond to executions of the 
system being model checked. To overcome this restriction, we propose a more general inference 
algorithm that uses finite generative structures (similar to automata) to represent families of 
explanations. The inference algorithm computes the probability of a possibly infinite set of 
explanations directly from the finite generative structure. We have implemented our inference 
algorithm in XSB Prolog, and use this implementation to encode probabilistic model checkers 
for a variety of temporal logics, including PCTL and GPL (which subsumes PCTL*). Our 
experiment results show that, despite the highly declarative nature of their encodings, the 
model checkers constructed in this manner are competitive with their native implementations. 

1 Introduction 

Beginning in 1997, we formulated the problem of model checking as one of query evaluation over 
logic programs [31] . The attractiveness of this approach is that the operational semantics of complex 
process languages (originally CCS [23], followed by value-passing calculi [32], the pi-calculus [23], 
and mobile calculi with local broadcast [3lJ), as well as the semantics of complex temporal logics 
(e.g., the modal mu-calculus [20J), can be expressed naturally and at a high level as clauses in a 
logic program. Model checking over these languages and logics then becomes query evaluation over 
the logic programs that directly encode their semantics. 

The past two decades have witnessed a number of important developments in Probabilistic 
Logic Programming (PLP), combining logical and statistical inference, and leading to a number 
of increasingly mature PLP implementations. A natural question is whether the advances in PLP 
enable the development of model checkers for probabilistic systems, the same way traditional LP 

*A prototype implementation of the techniques described in this paper is available at 
http : //www. cs . stonybrook.edu/~cram/probmc 
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3 "switches" (random processes) for transitions 
y, from states sO, si and s4, respectively. 
values(t(sO) , [sO, si, s2]). 
values(t(sl), [si, s3, s4] ) . 
values (t (s4) , [s3] ) . 

'/, Distribution parameters of the random variables. 
set_sw(t(sO) , [.5, .3, .2]). 
set_sw(t(sl) , [.4, .1, .5]). 
set_sw(t(s4) , [1]). 

y. Transition from S at instance I goes to T, 
y, as determined by the corresponding random process, 
trans (S, I, T) :- 

msw(t(S), I, T). 

y. Starting at state S at instance I, state T is reachable, 
reaches, I, T) :- 
transCS, I, U) , 
reachCU, next (I), T) . 
reaches, _, S) . 

(b) 

Figure 1: (a) Example Markov chain; (b) PRISM encoding of transitions in the chain. 

methods such as tabled evaluation and constraint handling enabled us to formulate model checkers 
for a variety of non-probabilistic systems. 

It turns out that existing PLP inference methods are not sufficiently powerful to be used as 
a basis for probabilistic model checking. One of the earliest PLP inference procedures, used in 
PRISM |40j . is formulated in terms of the set of explanations of answers. PRISM puts in place 
three restrictions to make its inference work: (a) independence: random variables used in any single 
explanation are all independent; (b) mutual exclusion: two distinct explanations of a single answer 
are mutually exclusive; and (c) finiteness: the number of possible explanations of an answer is 
finite. Subsequent systems, notably ProbLog [1] and PITA [35] have eliminated the independence 
and mutual exclusion restrictions of PRISM. This, however, is still insufficient for model checking, 
as the following example shows. 

Motivating Example: Figure [T] shows a Markov chain and its representation in PRISM. Note 
that the behavior of a Markov chain is memoryless: in any execution of the chain, a transition 
from state, say s, is independent of any previous transitions (including those from the same state). 
The definition of the trans predicate has an explicit instance parameter I, which is subsequently 
used in msw. PRISM treats different instances of the same random variable as independent. Thus 
trans correctly encodes the semantics of the Markov chain. 

We first consider simple reachability questions of the form: What is the likelihood that on an 
execution of the chain from a start state s, a final state t will be reached? The reachability question 
using the reach predicate is defined in Figure [T]^b) . Consider the likelihood of reaching state s3 
from sO. This query can be posed as the predicate prob(reach(sO, 0, s3) , P), where prob/2 
finds the probability of answers (P) to a given query reach(s0,0,s3). 

The query prob(reach(s0,0,s3) ,P) cannot be evaluated in PRISM. We illustrate this by first 
describing PRISM's inference at a high level. In PRISM, inference of probabilities proceeds in the 
same way as logical inference, except when the selected literal is an msw. In this case, the inference 
procedure enumerates the values of the random variable, and continues the inference for each value 
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(by backtracking). The probability of a derivation is simply the product of the probabilities of the 
random variables (msw outcomes) used in that derivation (under the independence assumption). 
The probability of a query answer is the sum of probabilities of the set of all derivations for that 
answer (using the mutual-exclusiveness and finiteness assumptions). Note that reach (sO , , s3) 
has infinitely many derivations, and hence PRISM cannot infer its probability. 

Markov chains can be encoded in ProbLog and LPAD [3^ in a similar manner. As is the case for 
PRISM, however, analogous reachability queries cannot be evaluated in these systems either. The 
sequence of random- variable valuations used in the derivation of an answer is called an explanation. 
In contrast to PRISM, ProbLog [1] and PITA [35], which is an implementation of LPAD, materialize 
the set of explanations of an answer in the form of a BDD. Probabilities are subsequently computed 
based on the BDD. This approach permits these systems to correctly infer probabilities even when 
the independence and mutual-exclusion assumptions are violated. Note that in the evaluation of 
reach, when a state is encountered, the next state is determined by a fresh random process. Hence, 
the set of explanations of reach (s0,0,s3) is infinitej^ Since BDDs can only represent finite sets, 
the probability of reach(s0,0,s3) cannot be computed in ProbLog or LPAD. 

To correctly infer the probability of reach(sO,0, s3) , we need an algorithm that works even 
when the set of explanations is infinite. Moreover, it is easy to construct queries where the in- 
dependence and mutual exclusion properties do not hold. For example, consider the problem of 
inferring the probability of reaching s3 or s4 (i.e., the query reach(s0,0,s3) ; reach(sO,0,s4)). 
Since some paths to s3 pass through s4, explanations for reach(s0,0,s3) and reach(sO , , s4) 
are not mutually exclusive. The example of Fig. [T] illustrates that to build model checkers based 
on PLP, we need an inference algorithm that works even when the finiteness, mutual-exclusion and 
independence assumptions are simultaneously violated. 

Summary of Contributions: In this paper, we present PIP (for "Probabilistic Inference Plus"), 
a new algorithm for inferring probabilities of queries in a probabilistic logic program. PIP is 
applicable even when explanations are not necessarily mutually exclusive or independent, and the 
number of explanations is infinite. We demonstrate the utility of this new inference algorithm 
by constructing model checkers for a rich class of probabilistic models and temporal logics (see 
Section [5]). Our model checkers are based on high-level, logical encodings of the semantics of the 
process languages and temporal logics, thus retaining the highly declarative nature of our prior 
work on model checking non-probabilistic systems. 

We have implemented our PIP inference algorithm in XSB Prolog [l2]. Our experimental 
results show that, despite the highly declarative nature of our encodings of the model checkers, 
their performance is competitive with their native implementations. 

The rest of this paper develops along the following lines. Section|3]provides requisite background 
on probabilistic logic programming. Section [4] presents our PIP algorithm. Section [5] describes 
our PLP encodings of probabilistic model checkers, while Section [6] contains our experimental 
evaluation. Section [7] offers our concluding remarks and directions for future work. 

'^This is in contrast to the link-analysis examples used in ProbLog and PITA |36) . where, even though the number 
of derivations for an answer may be infinite, the number of explanations is finite. 
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2 Related Work 



There is a substantial body of prior work on encoding complex model checkers as logic programs. 
These approaches range from using constraint handling to represent sets of states such as those 
that arise in timed systems [l2l [6l ETJ [29] , data- independent systems [39] and other infinite-state 
systems [5|f26]; tabling to handle fixed point computation [33', '8]; procedural aspects of proof search 
to handle name handling [47j and greatest fixed points [llj . However, all these works deal only 
with non-probabilistic systems. 

With regard to related work on probabilistic inference, Statistical Relational Learning (SRL) has 
emerged as a rich area of research into languages and techniques for supporting modeling, inference 
and learning using a combination of logical and statistical methods [lOj. Some SRL techniques, 
including Bayesian Logic Programs (BLPs) [18j, Probabilistic Relational Models (PRMs) [9] and 
Markov Logic Networks (MLNs) [3l], use logic to compactly represent statistical models. Others, 
such as PRISM [50], Stochastic Logic Programs (SLP) [25], Independent Choice Logic (ICL) [30], 
CLP(BN) [38j, ProbLog [4J, LPAD [44j and CP-Logic [l3], define inference primarily in logical 
terms, subsequently assigning statistical properties to the proofs. Motivated primarily by knowledge 
representation problems, these works have been naturally restricted to cases where the models and 
the inference proofs are finite. Recently, a number of techniques have generalized these frameworks 
to handle random variables that range over continuous domains (e.g. [19l [281 HHl HSl O US]), but 
still restrict proof structures to be finite. 

Modeling and analysis of probabilistic systems, both discrete- and continuous-time, has been 
an actively researched area. Probabilistic Computation Tree Logic (PCTL) |l5j is a widely used 
temporal logic for specifying properties of discrete-time probabilistic systems. PCTL* pj is a 
probabilistic extension of LTL and is more expressive than PCTL. Generalized Probabilistic Logic 
(GPL) [3J is a probabilistic variant of the modal mu-calculus. The Prism model checker |22j is 
a leading tool for modeling and verifying a wide variety of probabilistic systems: Discrete- and 
Continuous-Time Markov chains and Markov Decision Processes. There is also prior work on 
techniques for verifying more expressive probabilistic systems, including Recursive Markov chains 
(RMCs) [7] and Probabilistic Push-Down systems [51], both of which exhibit context-free behavior. 
The probability of reachability properties in such systems is computed as the least solution to a 
corresponding set of monotone polynomial equations. PReMo [46j is a model checker for RMCs. 
Reactive Probabilistic Labeled Transition Systems (RPLTS) [3] generalize Markov chains by adding 
external choice (multiple labeled actions). GPL properties of such systems are also computed as 
the least (or greatest, based on the property) solution to a set of monotone polynomial equations. 
To the best of our knowledge, this paper presents the first implementation of a GPL model checker. 



3 Preliminaries 

Notations: The root symbol of a term t is denoted by 7r(t) and its i-th subterm by argj(t). 
Following traditional LP notation, a term with a predicate symbol as root is called an atom. The 
set of variables in a term t is denoted by vars{t). A term t is ground if vars{t) = 0. 

Following PRISM, a probabilistic logic program (PLP) is of the form P = Pp U Pr, where Pr 
is a definite logic program, and Pp is the set of all possible msw/3 atoms. The set of possible 
msw atoms and the distribution of their subsets is given by values and set_sw directives, respec- 
tively. For example, clauses trans and reach in Fig. W(h) are in Pr. The set Pp of that program 
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contains msw atoms such as msw(t (sO) , 0, sO) , msw(t (sO) ,next (0) , sO) , msw(t (sO) , 0, si), 
. . ., msw(t (si) ,next (0) , si),.... 

In an atom of the form nisw(ii, ^s); is a term representing a random process (switch in 
PRISM terminology), t2 is an instance and is is the outcome of the process at that instance. 
According to PRISM semantics, two msw atoms with distinct processes are independent; and two 
msw atoms with distinct instances (even if they have the same process) are independent. Two msw 
atoms with the same process and instance but different outcomes are mutually exclusive. 

4 The Inference Procedure PIP 

A key idea behind the PIP inference algorithm is to represent the (possibly infinite) set of expla- 
nations in a symbolic form. Observe from the example in Fig. [T] that, even though the set of paths 
(each with its own distinct probability) from state sO to state s3 is infinite, the regular expression 
s0+sl*s4^s3 captures this set exactly. Following this analogy, we devise a grammar-based notation 
that can succinctly represent infinite sets of finite sequences. 

Definition 1 (Explanation) An explanation of an atom A with respect to a PLP P = Pp U Pr 
is a set ^ C Pp o/msw atoms such that (i) S,,Pji h A and (ii) ^ is consistent, i.e. it contains no 
pair of mutually exclusive msw atoms. 

The set of all explanations of A w.r.t. P is denoted by £p{A). □ 

Example 1 (Set of explanations) Consider the PLP of Fig. \^b). The set of explanations for 
reach(sO, 0, s3) is: 

msw(t(sO), 0, si), msw(t(sl), next{0), s3). 

msw(t(sO), 0, sO), msw(f(sO), next{0), si), msw(t(sl), next{next{0)), s3). 
msw(t(sO), 0, si), msw(f(sl), next{0), si), msw(t(sl), next{next{0)), s3). 
4.1 Representing Explanations 

As Example [T] illustrates, a representation in which instance identifiers are explicitly captured will 
not be nearly as compact as the corresponding regular expression (shown earlier). On the other 
hand, a representation (like the regular expression) that completely ignores instance identifiers will 
not be able to identify identical instances of a random process nor properly distinguish distinct 
ones. 

We solve this problem by observing that in PRISM's semantics, different instances of the 
same random process are independent and identically distributed (i.i.d.). Consequently, the prob- 
ability of reach (s0,0,s3) (reaching s3 from sO starting at instance O), is the same as that of 
reach(sO,next(0) ,s3) (starting at instance l), which is the same as that of reach(sO,H,s3), 
for any instance H. Consequently, it is sufficient to infer probabilities for a single parameterized 
instance. Below, we formalize the set of PLP programs for which such an abstraction is possible. 

Definition 2 (Temporal PLP) A temporal probabilistic logic program is a probabilistic logic pro- 
gram P with declarations of the form temporal (p/n — i) , where p/n is an n-ary predicate, and i 
is an argument position (between 1 and n) called the instance argument of p/n . Predicates p / n in 
such declarations are called temporal predicates. □ 
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The set of temporal predicates in a temporal PLP P is denoted by temporal (P); the set of all 
predicates in P is denoted by preds(P). By convention, every temporal PLP contains an implicit 
declaration temporal (msw/3-2), indicating that msw/3 is a temporal predicate, and its second 
argument is its instance argument. The instance argument of a predicate p/n is denoted by 
For example, the program of Fig. [T|b) becomes a temporal PLP when temporal (trans/3-2) and 
temporal (reach/3-2) are added. For this program, temporal(P) = {reach/3, trans/3, msw/3}, 
and x(reach/3) = x('trans/3) = x(msw/3) = 2. 

We extend the notion of instance argument from predicates to atoms as follows. Let a be an 
atom in a temporal PLP such that its root symbol is a temporal predicate, i.e., 7r(a) G temporal(P). 
Then the instance of a, denoted by x(q^) by overloading the symbol x, is arg^(-^(^))(a). We also de- 
note, by x(a), a term constructed by omitting the instance of q; i.e. if a = /(ti, . . . , tj-i, tj, tj+i, ■ ■ - tn) 
and x(a) = U, then = /(^i, . . • , tj-i, ti+i, . . . t„). 

Explanations of a temporal PLP can be represented by a notation similar to Definite Clause 
Grammars (DCGs). 

Example 2 (Set of explanations using DCG notation) Considering again the program of Fig.^b), 
the set of explanations for reach(sO,H, s3) can be succinctly represented by the following DCG: 



expl(reach(sO 
expl(reach(sO 
expl(reach(sl 
expl(reach(sl 
expl(reach(sl 
expl(reach(s3 
expl(reach(s4. 



s3),H) 
s3),H) 
s3),H) 
s3),H) 
s3),H) 
s3),H) 
s3),H) 



[msw(t(sO), H, sO)], expl(reach(sO, s3),next{H)) 
[msw(t(sO), H, si)], expl(reach(sl, s3),next{H)) 
[msw(t(sl), H, si)], expl(reach(sl, s3),next{H)) 
[msw(t(sl), if, s3)], expl(reach(sl, s3),next{H)) 
[msw(t(sl), i?, s4)], expl(reach(s4, s3),next{H)) 

[]• 

[msw(t(s4), H, s3)], expl(reach(s3, s3) , next{H)) . 



Note that each expl generates a sequence of msws. For this example, it is also the case that 
in a string generated from expl(reach(sO, s3), ff), the msws all have instances equal to or later 
than H. It is then immediate that msw(t(sO) , H, sO) is independent of any msw generated from 
expl(reach(sO, s3), next(-?/)). This property holds for an important subclass called temporally 
well-formed programs, defined as follows. 

Definition 3 (Temporally Well-Formed PLP) A temporal PLP P is said to be temporally 
well formed if for each clause (a :— f3i, . . . f3n) £ P: 

• If Ti{a) G temporal(P) then \/ i,l < i < n, s.t. 7r(/3j) G temporal(P), x(ft) contains x(q^); 
and vars{l3i) = vars{a). 

• If IT (a) temporal(P) then there is at most one i, 1 <i <n s.t. '7r(/3j) G temporal(P). 

• Instance arguments x(q^) or xiPi) or their subterms are unified only with other instance 
arguments, their subterms, or with ground terms. □ 



For temporally well- formed programs, the explanations for an atom can be represented suc- 
cinctly by DCGs. Such DCGs are called explanation generators. 
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Definition 4 (Explanation Generator) Let P he a temporally well-formed PLP and let Q he a 

query such that all non-instance- arguments of Q are ground (i.e. x{Q) ground). Then, an expla- 
nation generator for Q with respect to P is a DCG T , with non-terminals of the form expl(G, H) 
and terminals of the form insw(r, H, v) such that: 

• For every production (/3o — /3i, • ■ • , /3n) , \/ i, {) < i < n, all non-instance- arguments of Pi 
are ground; i.e., if (3i = expl(G, H), then G is ground, and if (3i = nisw(r, H, v), then r and v 
are ground. 

• £p{Q), the set of explanations for Q w.r.t. P, is identical to the language of T with ex-pl{xiQ),xiQ)) 
as the start symbol. □ 

The DCG in Example [2] is the explanation generator for the query reach(sO,H,s3) over the 
program given in Figure [T]^b) . In general, an explanation generator may not be in a form from 
which we can directly infer the probabilities. For this purpose, we use the factoring algorithm 
described below. 

4.2 Factored Explanation Diagrams 

The factored form of an explanation generator is obtained by constructing a Factored Explanation 
Diagram (FED), whose structure closely follows that of a BDD. Similar to a BDD, a FED is 
a labeled direct acyclic graph with two distinguished leaf nodes: tt, representing true, and ff , 
representing false. While the internal nodes of a BDD are Boolean variables, a FED contains two 
kinds of internal nodes: one representing terminal symbols of explanations (msws), and the other 
representing non-terminal symbols of explanations (expls). We use a partial order among nodes, 
denoted by "<", to construct a FED. 

Definition 5 (Factored Explanation Diagram) A factored explanation diagram (FED) is a 
labeled directed acyclic graph with: 

• Four kinds of nodes: tt, f f , msw(r, h) and expl(t, h), where r is a ground term representing 
a random process, t is a ground term, and h is an instance term; 

• Nodes tt and f f are 0-ary, and occur only at leaves of the graph; 

• insw(r, h) is an n-ary node when r is random process with n outcomes, and the edges to the n 
children are labeled with the possible outcomes of r; 

• expl(t, h) is a binary node, and the edges to the children are labeled and 1. 

• // there is an edge from node xi to X2, then xi < X2. □ 

Note that the multi- valued decision diagrams used in the implementation of PITA |i36] are a special 
case of FEDs with only tt, f f and msw(r, h) nodes, where r and h are ground. 

We represent non-trivial FEDs by x?Alts, where x is the node and Alts is the list of 
edge-label/child pairs. For example, a FED F whose root is an MSW node is written as 
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iiisw(r, h)7[vi:Fi,V2-F2, . . . , where Fi, F2, . . . , F„ are children FEDs (not all necessarily dis- 

tinct) and vi,V2, ■ ■ ■ ,Vn are possible outcomes of the random process r such that Vi is the la- 
bel on the edge from F to Fi. Similarly, a FED F whose root is an EXPL node is written as 
expl(t, /i)?[0:Fo, l:Fi], where Fq and Fi are the children of F with edge labels and 1, respectively. 

We now define the ordering relation "<" among nodes. We first define a time order among 
instances such that hi -< /i2 if hi represents an earlier time instant than h2- If hi /12 and /12 2^ ^i; 
then hi and /12 are incomparable, denoted as /ii ~ /i2. We also assume an arbitrary order < among 
terms. 

Definition 6 (Node order) Let xi and X2 be nodes in a FED. Then xi < X2 if it matches one 
of the following cases: 

• msw(ri,/ii) <msw(r2,/i2) if hi -< /i2 or (ri < r2 and (hi = /12 or hi ~ h2)) 

• msw(ri, hi) < expl(t2, /12) if hi -< /i2 or hi ~ /12 

• expl(ti,/ii) < expl(t2,/i2) if ti < t2 and hi ~ /i2. □ 

Proposition 1 (Independence and Node Order) For any nodes xi, X2 in an FED, if xi < X2 
or X2 < xi, then xi and X2 are independent. 

Definition 7 (Binary Operations on FEDs) Fi © F2, where ® G {A, V} is a FED F derived 
as follows: 

• Fi is tt, and ® = V, then F = tt. 

• Fi is tt, and © = A, then F = F2. 

• _Fi is fi, and ® = A, then F = ff . 

• Fi is ff , and © = V, then F = F2. 

• Fi= xi?[i;i,i:Fi,i, . . . , ui,„i:Fi,„J, F2 = X2?[t;2,i:F2,i, . . ■ ,V2,n2-F2,n2]: 

Cl. Xi <X2: F = xi?[vix-{Fi^i ® F2), . . . ,Vi^nAFl,ni ©^2)] 
c2. xi=X2: F = xi?[vi,i:(Fi,i©F2,i),...,?;i,„,:(Fi,„, ©F2,„,)] 

C3. Xi>X2: F = X2?[v2,liFl®F2,l),---,V2,n2-{Fl®F2,n2)] 

c4. xi X2,X2 ft xi: F = expl (merge (©, Fi, F2), /i)?[0:ff, l:tt] where h is the common 
part of instances of xi and X2. Q 

Note that Def. [7]is a generalization of the corresponding operations on BDDs. Also, when xi and X2 
are both msw nodes, since < defines a total order between them, case c4 will not apply. When the 
operand nodes cannot be ordered (case c4), we generate a placeholder (a merge node) indicating 
the operation to be performed. Such placeholders will be expanded when FEDs are built from an 
explanation generator (see Def. [8] below). Note that merge nodes may be generated only if one or 
both arguments is a FED rooted at an expl node. 

We now give a procedure for constructing FEDs from an explanation generator for query Q 
with respect to program P. 
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Definition 8 (Construction of Factored Explanation Diagrams) Given an explanation 
generator T, F is a FED corresponding to goal G if fed{G, F) holds, where fed_c is the smallest 
relation and E is the smallest set such that: 

• fed_c(^o,-^) holds whenever {(/3o /3i,i, . . . , /3i,„J, (/3o /32,i, • • • , /32,n2); (/^o 
/Sfc^i, . . . ,/3fc,nfe)} is the set of all clauses in P with /3o on the left hand side, and 



• f ed_c(/3o, holds whenever (Sq = merge (©, Fi, F2) G i?, and 

— Fi = expl(ii,/ii)?[0:Fi,o, f ed(expl(ti, /ii), F{) holds, 
and F = ©(F{[ff ^ Fi,o,tt ^ Fi,i],F2). 

— F2 = expl(t2,/i2)?[0:F2,o, l:i^2,i]; fed(expl(t2,/i2),i^2) /^oWs, 
and F = ©(Fi,F^[ff F2,o,tt 1-^ ^2,1]). 

• fed(G, F) holds whenever 

— G = msw(r, and F = iasv(r,h)?[vi:Fi, . . .Vn'-Fn] where for all i, Fi = tt if Vi = v 
and Fi = f f otherwise. 

— G = expl{t,h), h is neither a ground term nor a variable, 
and F = expl(t, /i)?[0:f f , l:tt] . 

— G = expl{t,h), h is either ground or a variable, and F = Vfg^j c(G F')-^' ■ 

• merge (©, Fi, F2) G E whenever there is some G,F such that fed{G,F) holds, and there is a 
node in F of the form expl (merge (©, Fi,F2),h). □ 

The above definition is inductive, and can be turned into a tabled logic program implementing 
the construction procedure. Furthermore, FEDs are maintained using a dictionary to ensure that 
the FEDs have a DAG instead of tree structure. 

Example 3 Three of the four FEDs for the explanation generator in Example are shown in 
Fig.^ The FED for expl (reach (s3, s3) ,H) , not shown in the figure, is tt. 

4.3 Computing Probabilities from FEDs 

A factored explanation diagram can be viewed as a stochastic grammar. Following [7], we can 
generate a set of simultaneous equations from the stochastic grammar, and find the probability 
of the language from the least solution of the equations. The generation of equations from the 
factored representation of explanations is formalized below. 

Definition 9 (Temporal Abstraction) Given a temporal PLP P, the temporal abstraction of a 
term t, denoted by abs(t) is x{t) if T^it) £ temporal(P), and x{i) is non-ground; and t otherwise. 
That is, for a term t with a temporal predicate as root, abs(t) replaces its instance argument with 
a special symbol _L if that argument is not ground. □ 




k 




) holds 



i=i j=i 
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msw(t(sO), H) 



expl (reach (sO, s3) , expl (reach (si, s3} , 

next (H) ) next (H) ) 




(a) FED for expl (reach (s0,s3) ,H) 



msw(t(sl), H) 




insw(t(s4), H) 



expl (reach (si, s3) , expl (reach (s3, s3} , expl (reach(s4, s3) , 



next (H) ) 



next (H) ) 



next (H) ) 



expl (reach(s3, s3) , next (H) ) 





(b) FED for expl(reach(sl,s3) ,H) 
(c) FED for expl (reach (s4,s3) ,H) 

Figure 2: FEDs for Example [2] 



Definition 10 (Distribution) Let p he a random process specified in a PLP P. The set of 

values produced by p is denoted by valuesp(/9). The distribution of p, denoted by dlstr p{p) , 
is a function from the set of all terms over the Herbrand Base of P to [0, 1] such that 
E^6valuesp(p) distrp(p)(^;) = 1 □ 

Definition 11 (System of Equations for PLP) Let T be an explanation generator, fed be 
the relation defined in Def. \^ V be a countable set of variables, and f be a one-to-one func- 
tion from terms to V . The system of polynomial equations E(Yyj-^ = {(/(abs(G)) = V{F)) \ 
fed(G,F) holds}, where V is a function that maps FEDs to polynomials, is defined as follows: 



V{tt) 

P(msw(r, h)l[vi:Fi, . . . ,Vn.Fn]) 
P(expl(t,/i)?[0:Fo,l:Fi]) 



Er=idistr(r)(r;)*P(F,) 
/(abs(expl(t,/i)))*P(Fi) 

+ (l-/(abs(expl(t,/i))))*P(Fo) 



The set of equations for Example |3] is shown in Fig. [3| 

The implementation of the above definition is such that shared FEDs result in shared variables 
in the equation system, thereby ensuring that every FED is evaluated at most once. The correspon- 
dence between a PLP in factored form and the set of monotone equations permits us to compute 
the probability of query answers in terms of the least solution to the system of equations. 

Theorem 2 (Factored Forms and Probability) Let T he an explanation generator for query 
Q w.r.t. program P. Let V be a set of variables and let f be a one-to-one function from terms to 
V. Then, X is the probability of a query answer Q evaluated over P, denoted by proh{Q,X), if 
X is the value of the variable /(expl(x(Q), least solution of the corresponding set of 

equations, E(Yyjy 
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Xo = too *xo + toi * Xi too = -5 tl4 = -5 

Xi = til *Xi+ ti3 * X3 + ti4 * X4 toi = -3 t43 = 1 

X3 = 1 tii = A 

X4 = t43 * X3 ^13 = -1 

Figure 3: Set of equations generated from the set of FEDs of Example [3] 

The proof of the above theorem can be obtained by treating the explanations in P as strings 
generated by a corresponding stochastic CFG. Such a correspondence is possible since the explana- 
tions are represented in factored form. The following properties show that the algorithm for finding 
probabilities of a query answer is well defined. 

Proposition 3 (Monotonicity) IfT is a definite PLP in factored form, V is a set of variables 
and f is a one-to-one function as required by Def. IP then the system of equations E(Yyj-^ is 
monotone in [0, 1]. 

Monotone systems have the following important property: 

Proposition 4 (Least Solution [7J) Let E be a set of polynomial equations which is monotone 
in [0,1]. Then E has a least solution in [0,1]. Furthermore, a least solution can be computed to 
within an arbitrary approximation bound by an iterative procedure. 

Note that FEDs are non-regular since expl nodes may have other expl nodes as children, and 
hence the resulting equations may be non- linear. Proposition |4] establishes that probability of query 
answers can be effectively computed even when the set of equations is non- linear. 

The probability of the language of explanations in Example [2] (via the equations in Fig. [3]) is 
given by the value of xq in the least solution, which is 0.6. 



5 Applications 

We now present two model checkers that demonstrate the utility of the new PIP inference technique. 

PCTL: The syntax of an illustrative fragment of PCTL is given by: 

SF ::= prop(A) | neg(5F) | and(5Fi, ^Fa)! pr(PF,gt,S) | pr(PF,geq,B) 
PF ::= until(5Fi, SF2) \ next{SF) 

Here, ^ is a proposition and B is a real number in [0, 1]. The logic partitions formulae into state 
formulae (denoted by SF) and path formulae (denoted by PF). State formulae are given a non- 
probabilistic semantics: a state formula is either true or false at a state. For example, formula 
prop(a) is true at state s if proposition a holds at s; a formula and(S'Fi, SF2) holds at s if both 
SFi and SF2 hold at s. The formula pt{PF, gt, B) holds at a state s if the probability p of the set 
of all paths on which the path formula PF holds is such that p > B (similarly, p > B for geq). 

The formula until(S'Fi, 5*^2) holds on a single given path so, si, S2, . . . if SF2 holds on state Sk 
for some k > 0, and SFi holds for all Sj, < i < fc. Full PCTL has a bounded until operator, which 
imposes a fixed upper bound on k; we omit its treatment since it has a straightforward non-fixed- 
point semantics. The probability of a path formula PF at a state s is the sum of probabilities of all 
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% State Formulae 
models (S, prop (A)) :- 

holds (A) . 
models (S, neg(A)) :- 

not models (A) . 
models (S, and(SFl, SF2):- 

modelsCS, SFl) , 

models (S, SF2) . 
models (S, pr(PF, gt , B)) :- 

prob(pmodels(S, PF) , P) , 

P > B. 

models (S, pr(PF, geq, X)) :- 
prob(pmodels(S, PF) , Y) , 
P >= B. 



% Path Formulae 
pmodels(S, PF) :- 
pmodels(S, PF, 



.). 



:- table pmodels/3. 
pmodels(S, until (SFl, SF2) , H) 

models (SF2) . 
pmodels(S, until (SFl, SF2) , H) 

models (SFl) , 

trans (S, H, T) , 

pmodelsd, until (SFl, SF2) 
pmodels(S, next(SF), H) :- 

trans (S, H, T) , 

models (T, SF) . 



next(H)) 



temporal (pmodels/3-3) . 
Figure 4: Model checker for a fragment of PCTL 



paths starting at s on which PF holds. This semantics is directly encoded as the probabilistic logic 
program given in Fig. [4j Observe that the program is temporally well formed. Moreover, observe 
the use of an abstract instance argument "_" the invocation of pmodels/3 from piiiodels/2. This 
ensures that an explanation generator can be effectively computed for any query to pmodels/2. 

GPL: GPL is an expressive logic based on the modal mu-calculus for probabilistic systems [3]. 
GPL subsumes PCTL and PCTL* in expressiveness. GPL is designed for model checking reactive 
probabilistic transition systems (RPLTS), which are a generalization of DTMCs. In an RPLTS, a 
state may have zero or more outgoing transitions, each labeled by a distinct action symbol. Each 
action has a distribution on destination states. 

Syntactically, GPL has state and fuzzy formulae, where the state formulae are similar to those 
of PCTL. The fuzzy formulae are, however, significantly more expressive. The syntax of GPL, in 
equational form, is given by: 

SF ::= prop(A) | neg(prop(A)) | and(5F, SF) \ ot{SF, SF) 

I pr(PF, gt, B) I pr(PF, It, B) \ pr(PF, geq, B) \ pT{PF, leq, B) 

PF ::= sf(5F) I form(X) I and(PF,PF) I or(PF,PF) I diam(^,PF) I box(A,PF) 

D ::= def(X,lfp(PP)) I def(X,gfp(PP)) 

Formula diam(A, PF) holds at a state if there is an ^-transition after which PF holds; box(^, PF) 
holds at a state if PF holds after for every A-transition. Least- and greatest-fixed-point formulas 
are written as a definition D using Ifp and gfp, respectively. Formulae are specified as a set of 
definitions. GPL admits only alternation-free fixed-point formulae. 

A part of the model checker for GPL that deals with fuzzy formulae is shown in Fig. [5j 
Note that fuzzy formulae have probabilistic semantics, and, at the same time, may involve 
conjunctions or disjunctions of other fuzzy formulae. Thus, for example, when evaluating 
models(s, and(PPi, PF2), H), the explanations of models(s, PPi, H) and models(s, PP2, H) may 
not be pairwise independent. Thus recursion-free fuzzy formulae cannot be evaluated in PRISM, 
but can be evaluated using the BDD-based algorithms of ProbLog and PITA. In contrast, recursive 
fuzzy formulae can be evaluated using PIP. 
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%% pmodels(S, PF, H) : S is in the model of fuzzy formula PF at or after instant H 
%% smodels(S, SF): S is in the model of state formula SF 



pmodelsCS, sf(SF), H) :- 

smodels(S, SF) . 
pmodelsCS, and(Fl,F2), H) :- 

pmodelsCS, Fl, H) , 

pmodelsCS, F2, H) . 
pmodelsCS, orCFl,F2), H) :- 

pmodelsCS, Fl, H) ; 

pmodelsCS, F2, H) . 
pmodelsCS, diamCA, F) , H) :- 

trans CS, A, SW) , 

mswCSW, H, T), 

pmodelsCT, F, [T,SW|H] ) . 
pmodelsCS, boxCA, F) , H) :- 

findallCSW, trans CS , A, SW) , L), 

all_pmodelsCL, S, F, H) . 



pmodelsCS, formCX) , H) :- 

tabled_pmodelsCS, X, HI), H=H1 . 

all_pmodels C [] , _, _, _H) . 

all_pmodelsC [SWiRest] , S, F, H) :- 
mswCSW, H, T), 
pmodelsCT, F, [T,SW|H] ) , 
all_pmodels CRest , S, F, H) . 

:- table tabled_pmodels/3 . 
tabled_pmodelsCS,X,H) :- 

fdefCX, IfpCF)), 

pmodelsCS, F, H) . 



Figure 5: Model checker for fuzzy formulas in GPL 



Recursive Markov Chains: A Recursive Markov Chain (RMC) consists of components, which 
are analogous to procedure definitions in a procedural language. The structure of each component 
is similar to an automaton, with the addition of boxes that represent procedure calls. An RMC can 
be considered as an extension of DTMCs with recursively-called components. An example RMC 
from is shown in Fig.[6| There are four special node types in an RMC: entry (en) and exit nodes 
(ex) associated with components and call and return ports associated with boxes. In a box, call and 
return ports correspond to entry and exit nodes, respectively, of the called component. Behaviors 
of an RMC are the set of runs with matching calls and returns. Hence behaviors of an RMC form 
a context free language. We pose the problem of reachability in an RMC (i.e. the probability of 
the set of runs that hit a given state) in terms of GPL model checking of a corresponding RPLTS. 

Given an RMC R, in which the maximum number of exits in any component is n, we define an 
RPLTS R' and a set of n mutually recursive GPL formulae Xi,X2, ■ ■ ■ , Xn- R' will have a state for 
every node of R, in particular including the call and return ports of each box. For each probabilistic 
transition in R, we add the same transition in R' and label it as p. To model the recursive call. 



A A' 




Figure 6: Example of a Recursive Markov Chain (RMC) 
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Figure 7: RPLTS corresponding to the example RMC in Fig. |6] 



we introduce three types of transitions. From the state s in R' corresponding to a call port in R, 
we add a c (call) transition to state s' corresponding to the called component's entry node. We 
also add rj (return) transitions from state s to the states corresponding to the return ports of the 
call. Finally, from state s in R' corresponding to an exit node exi, we add an ej (exit) transition 
back to s. The exit transitions and their labels enable us to write GPL formulae that check for 
termination. The RPLTS corresponding to the example RMC in Fig. [6] is shown in Fig. [7} Labels 
on probabilistic transitions {p) are omitted in Fig. [7] to avoid clutter. 

We use GPL formulae Xi to specify behaviors in an RMC that eventually reach an exit node 
exi of a component. For a 2-exit RMC, the definitions of Xi are as follows: 

def (Xi, If p(or(diam(ei, tt), 
or(diam(p, Xi), 

or(aiid(diam(c, Xi), diain(ri, Xi)), 
and(diam(c, X2), diam(r2, Xi))))))). 
def (X2, If p(or(diam(e2, tt), 
or(diain(p, X2), 

or(and(diam(c, Xi), diain(ri, X2)), 
and(diam(c, X2), diam(r2, X2))))))). 

The intuition behind the formulae for Xi is as follows. Since Xi specifies that exi is eventually 
reached, Xi is a least fixed point formula. The ways in which exit exi is reached from a state s are: 

1. Cj transition is enabled at s: this corresponds to the disjunct diam(ej, tt) in the definition of 
Xi] 

2. there is a probabilistic transition (p) from s to s' such that exi is reached eventually from s' 
(corresponds to (diam(p, Xj)); 

3. s corresponds to a call, that call eventually returns from exj for some j, and subsequently, 
eXi is reached. The formula and(diam(c, Xj), diam(rj, Xj)) encodes this way of reaching eXi 
via exj. The subformula diam(c, Xj) specifies that exj is reached after the call; and the sub- 
formula diani(rj,Xj) specifies that after the corresponding return, exi is eventually reached. 

While the example shows the GPL formulae for 2-exit RMCs, the description above gives the 
general structure of the formulae for n-exit RMCs. Note that if a component has fewer than n 



14 



exits, then the formula X„ will be trivially false at all of its nodes. Moreover, behaviors satisfying 
Xi and those satisfying Xj (i ^ j) are mutually exclusive, since we cannot terminate at more than 
one exit on a single path. Finally, the GPL formula is the same regardless of what RMC we are 
attempting to transform, and only depends on n. 



6 Experimental Results 

PIP has been implemented using the XSB tabled logic programming system |42j. An explanation 
generator is constructed by performing normal query evaluation under the well-founded semantics 
by redefining msws to backtrack through their potential values, and have the undefined truth value. 
This generates a residual program in XSB that captures the dependencies between the original goal 
and the msws (now treated as undefined values). In one partial implementation, called PIP-Prism, 
the probabilities are computed directly from the residual program. Note that such a computation 
will be correct if PRISM's restrictions are satisfied. In general, however, we materialize the residual 
program into a dynamic database that corresponds to the productions in the explanation generator. 
A second partial implementation, called PIP-BDD, constructs BDDs from the explanation gen- 
erator, and computes probabilities from the BDD. Note that PIP-BDD will be correct when the 
finiteness restriction holds. The full implementation of PIP, called PIP-full, is obtained by con- 
structing a set of FEDs from the explanation generator (Def. ^ , generating polynomial equations 



from the set of FEDs (Def. 11) and finally finding the least solution to the set of equations. The 
final equation solver is implemented in C. All other parts of the three implementations, including 
the BDD and FED structures, are completely implemented in tabled Prolog. 

We present two sets of experimental results, evaluating the performance of PIP on (1) programs 
satisfying PRISM's restrictions; and (2) a program for model checking PCTL formulae. 

Performance on PRISM Programs: Note that all three implementations — PIP-Prism, 
PIP-BDD and PIP-full may be used to evaluate PRISM programs. 



Hidden Markov Model (HMM): We used the simple 2-state gene sequence HMM from [2] 
(also used in [37j) for our evaluation. We measured the CPU time taken by the versions of PIP, 
PRISM 2.0.3 and PITA-INDEXC [37] (a version of PITA that does not use BDDs and uses PRISM's 
assumption) to evaluate the probability of a given observation sequence, for varying sequence 
lengths. The observation sequence itself was embedded as a set of facts (instead of an argument 
list). This makes table accesses fast even when shallow indices are used. The performance of 
the three PIP versions and PITA-INDEXC, relative to PRISM 2.0.3, is shown in Fig.[8]^a). CPU 
times are normalized using PRISM's time as the baseline. Observe that the PIP-Prism and PITA- 
INDEXC perform similarly: about 3.5 to 4 times slower than PRISM. Construction of BDDs 
(done in PIP-BDD, but not in PIP-Prism) adds an extra factor of 4 overhead. Construction of 
full-fiedged FEDs, generating polynomial equations and solving them (done only in PIP-full) adds 
another factor of 2 overhead. We find that the equation-solving time (using the only component 
coded in C) is generally negligible. 



Probabilistic Left Corner Parsing: This example was adapted from PRISM's example suite, 
parameterizing the length of the input sequence to be parsed. We measured the CPU time taken by 
the three versions of PIP and PRISM 2.0.3 on a machine with an Intel Pentium 2.16GIIz processor. 
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PRISM ■ 

PIP-Prism - 

PIP-BDD ■ 

PIP-full ■ 



1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 
Length of Observation Sequence 

(a) Relative performance on HMM 



Length of Sentence 



(b) Performance for NCN queries for PLC 




PRISM ■ 

PIP-Prism - 

PIP-BDD ■ 

PIP-full ■ 




Length of Sentence Length of Sentence 

(c) Performance for NPV queries for PLC (d) Performance for ADVN queries for PLC 
Figure 8: Performance of PIP on PRISM Programs 





(a) N 



No. of slots 
: 5 



(h) N = 6 



Figure 9: Performace of PCTL model checking using PIP and the Prism model checker for Syn- 
chronous Leader Election protocol of different sizes 



The performance on three queries (each encoding a different class of strings) is shown in Fig. ^ 
(d). In contrast to the HMM example, the sequences are represented as lists. For these examples, 
PIP-Prism implementation outperforms PRISM. Moreover, although PIP-BDD and PIP-full are 
slower than PIP-Prism, the relative performance gap is much smaller than observed in the HMM 
example. 

Performance of the PCTL Model Checker: We evaluated the performance of PIP-full for 

supporting a PCTL model checker (encoded as shown in Fig. |4]). We compared the performance 
of PIP-based model checker with that of the widely- used Prism model checker \22\ . We show the 
performance of PIP and the Prism model checker on the Synchronous Leader Election Protocol |17j 
for computing the probability that eventually a leader will be elected. Fig. [9] shows the CPU time 
used to compute the probabilities of this property on systems of different sizes. Observe that our 
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high-level implementation of a model checker based on PIP performs within a factor of 3 of the 
Prism model checker (note: the y-axis on these graphs is logarithmic). Moreover, the two model 
checkers show similar performance trends with increasing problem instances. However, it should 
be noted that the Prism model checker uses a BDD-based representation of reachable states, which 
can, in principle, scale better to large state spaces compared to the explicit state representation 
used in our PIP-based model checker. 

7 Conclusions 

In this paper, we have shown that in order to formulate the problem of probabilistic model checking 
in probabilistic logic programming, one needs an inference algorithm that functions correctly even 
when finiteness, mutual-exclusion, and independence assumptions are simultaneously violated. We 
have presented such an inference algorithm, PIP, implemented it in XSB Prolog, and demonstrated 
its practical utility by using it as the basis for encoding model checkers for a rich class of probabilistic 
models and temporal logics. 

For future work, we plan to refine and strengthen the implementation of PIP. We also plan to 
explore more substantial model-checking case studies. It would be interesting to study whether 
optimizations to exploit data independence and symmetry, which are easily enabled by high-level 
encodings of model checkers, will be effective for probabilistic systems as well. 

Acknowledgments. Research supported in part by NSF Grants CCF-1018459 CCF-0926190, 
CCF-0831298, AFOSR Grant FA9550-09- 1-0481, and ONR Grant N00014-07-1-0928. 
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